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Accomplishments 

The  goal  of  this  project  was  to  construct  models  for  decision  making  and  game  theory  that  can 
deal  with  complex  state  spaces,  resource-bounded  and  cognitively-limited  agents,  and  unfore¬ 
seen  contingencies  in  a  robust  way. 

A  number  of  advances  were  made  on  this  front,  outlined  below.  Much  of  the  work  reported 
below  is  summarized  in  [4],  which  has  been  given  as  an  invited  talk  at  numerous  conferences 
and  universities.1 


Taking  computation  into  account  in  game  theory:  Nash  equilibrium  does  not  take 
computation  into  account.  To  see  why  this  might  be  a  problem,  consider  the  following  example, 
taken  from  [29,38]. 

Example  1:  You  are  given  an  n-bit  number  x.  You  can  guess  whether  it  is  prime,  or  play  safe 
and  say  nothing.  If  you  guess  right,  you  get  $10;  if  you  guess  wrong,  you  lose  $10;  if  you  play 
safe,  you  get  $1.  There  is  only  one  Nash  equilibrium  in  this  1-player  game:  giving  the  right 
answer.  But  if  n  is  large,  this  is  almost  certainly  not  what  people  will  do.  Even  though  primality 
testing  can  be  done  in  polynomial  time,  the  costs  for  doing  so  (buying  a  larger  computer,  for 
example,  or  writing  an  appropriate  program),  will  probably  not  be  worth  it  for  most  people. 
The  point  here  is  that  Nash  equilibrium  is  not  taking  the  cost  of  computing  whether  x  is  prime 
into  account.  | 

There  have  been  attempts  in  the  game  theory  community  to  define  solution  concepts  that 
take  computation  into  account;  we  provide  one  that  greatly  generalizes  an  earlier  approach  due 
to  Rubinstein.  We  consider  Bayesian  games ,  where  each  player  has  a  type,  which  can  be  viewed 
as  the  player’s  private  information.  (In  the  example  above,  this  would  be  the  n-bit  number.) 

1A  partial  listing  is  given  in  the  section  on  invited  presentations  below. 
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The  players  choose  a  Turing  machine  (TM)  that  plays  for  them.  The  input  to  the  TM  is  the 
player’s  type.  We  associate  with  each  TM  and  input  a  complexity.  We  then  allow  a  player’s 
utility  to  depend  on  the  profile  of  moves  played  (the  outputs  of  each  player’s  TM),  the  profile  of 
types,  and  the  profile  of  complexities.  Thus,  we  can  “charge”  for  a  TM  that  takes  a  long  time 
to  run,  or  uses  a  lot  of  space,  on  a  given  input.  We  can  also  charge  for  the  complexity  of  the  TM 
(i.e.,  the  number  of  states  in  the  TM);  this  corresponds  to  how  complicated  a  player’s  program 
is.  Moreover,  we  can  take  a  given  TM  to  be  free,  but  charge  for  all  others.  This  amounts  to 
charging  for  searching  for  a  different  solution,  and  can  be  viewed  as  a  way  of  capturing  people’s 
reluctance  to  switch  from  a  strategy  they  are  already  comfortable  with. 

Formally,  a  Bayesian  machine  game  G  is  a  tuple  ([m],  M.,T,  Pr,  C\, ... . .  ,Cm,  u\, .. . . , um), 
where 


•  [m]  =  { 1,  -  -  • ,  rn}  is  the  set  of  players; 

•  M  is  the  set  of  possible  TMs; 

i  T  C  ({0,  l}*)m+1  ig  the  set  of  type  profiles,  where  the  ( m  +  l)st  element  in  the  profile 
corresponds  to  nature’s  type; 

•  Pr  is  a  distribution  on  T ; 

•  C,  is  a  complexity  function; 

•  Ui  :  T  x  ({0,  l}*)m  x  JNm  — >  1R  is  player  i’s  utility  function.  Intuitively,  Ui(t,a,c)  is  the 
utility  of  player  i  if  t  is  the  type  profile,  a  is  the  action  profile  (where  we  identify  i’s  action 
with  Mi  s  output),  and  c  is  the  profile  of  machine  complexities. 

A  strategy  for  player  i  is  a  function  from  7)  to  A  (A;)  (where,  as  usual,  we  denote  by  A(X) 
the  set  of  distributions  on  the  set  A).  If  a  is  a  strategy  for  player  i,  t  E  Tt  and  a  £  A;,  then 
a(t)(a)  denotes  the  probability  of  action  a  according  to  the  distribution  on  acts  induced  by 
cr{t).  Given  a  joint  strategy  a .  we  can  take  to  be  the  random  variable  on  the  type  space 
T  defined  by  taking  uf(t)  =  Z)aeJ4(°'i(^i)(ai)  x  . . .  x  am (tm)(am)) u% (t,  a).  Player  i’s  expected 
utility  if  <7  is  played,  denoted  Ui(a ),  is  then  just  Epr[uf]  =  YlfeT  Pr(i)wf(i). 

With  these  definitions  in  hand,  we  can  define  computational  Nash  equilibrium  in  the  standard 
way:  a  machine  profile  M  £  Aim  is  a  Nash  equilibrium  if,  for  all  players  i  and  TMs  M[  £  Ad, 


Thus,  for  each  player  i,  TM  Mt  is  a  best  response  to  M_*.  We  can  show  that,  in  general,  a 
computational  Nash  equilibrium  does  not  exist.  This  is  because  we  do  not  allow  mixing  over 
TMs.  While  TMs  can  randomize,  we  want  to  allow  “charging”  for  such.  If  we  do  not  charge  for 
randomization,  then  it  can  be  shown  that  a  computational  Nash  equilibrium  exists,  provided 
that  the  utility  functions  and  the  type  distribution  are  computable  [29,38]. 

In  [29,38],  the  framework  is  used  to  give  simple  explanations  of  phenomena  such  as  cooper¬ 
ation  in  repeated  Prisoner’s  Dilemma.  It  is  also  shown  that  there  are  deep  connections  between 
computational  games  and  cryptographic  protocol  security.  In  [41] ,  the  framework  is  applied  to 
decision  theory.  It  is  shown  that  the  approach  can  be  used  to  explain  well-known  phenomena 
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such  as  the  status  quo  bias  (people  are  much  more  likely  to  stick  with  what  they  already  have) 
and  belief  polarization  (after  hearing  the  exactly  same  information,  two  people  with  possibly 
different  prior  beliefs  can  end  up  with  diametrically  opposed  conclusions).  The  framework  is 
also  used  to  define  two  extensions  of  the  standard  notion  of  value  of  information:  value  of 
computational  information  and  value  of  conversation. 

Awareness:  In  game  theory,  it  is  typically  assumed  that  the  game  (i.e.,  the  agents  who  are 
playing,  the  moves  that  each  one  can  make,  and  the  payoffs  at  the  leaves)  is  common  knowledge: 
all  the  agents  know  them,  they  know  they  know,  and  so  on.  But  this  assumption  of  common 
knowledge  is  not  reasonable  in  general.  In  large  auctions  on  the  internet,  players  may  certainly 
not  know  how  many  other  players  there  are  in  the  game,  let  alone  who  they  are.  There  has 
been  a  great  deal  of  work  on  awareness  in  the  AI  and  game  theory  literatures,  going  back 
to  the  original  paper  written  by  Fagin  and  Halpern  (Belief,  awareness,  and  limited  reasoning, 
Artificial  Intelligence  34,  1988,  pp.  39-76).  None  of  the  earlier  work  dealt  with  knowledge  of 
lack  of  awareness  (indeed,  some  of  the  approaches  used  in  game  theory  are  provably  incapable 
of  dealing  with  it).  This  is  particularly  important  if  we  interpret  lack  of  awareness  as  stemming 
from  lack  of  computational  power.  An  agent  might  well  know  that  another  might  have  more 
computational  abilities  than  he  does,  and  so  know  things  that  he  is  not  aware  of  (because  he 
cannot  compute  them).  Rego  and  I  described  and  provided  a  complete  exiomation  of  a  logic 
that  could  capture  knowledge  of  lack  of  awareness  [27],  using  quantification  over  propositions. 
The  idea  was  that  you  could  say  “there  is  something  I  am  not  aware  of,  although  I  don’t  know 
what  it  is”. 

Unfortunately,  the  logic  has  a  significant  problem:  it  is  impossible  for  an  agent  to  be  uncer¬ 
tain  about  whether  he  is  aware  of  all  formulas.  This  problem  is  corrected  in  [28].  The  solution 
involves  considering  models  where  there  are  different  languages  associated  with  each  world; 
thus,  the  agent  has  uncertainty  about  what  the  laguage  is.  This  approach  allows  us  to  unify 
and  extend  much  of  the  earlier  work  on  awareness. 


Regret  minimization:  Perhaps  the  most  common  solution  concept  considered  in  game  the¬ 
ory  is  Nash  equilibrium.  While  many  other  solution  concepts  have  been  considered  in  the 
literature,  there  are  a  number  of  well-known  games  where  none  of  them  seems  appropriate. 

Example  2:  Consider  the  well-known  Traveler’s  Dilemma  (K.  Basu,  The  traveler’s  dilemma: 
paradoxes  of  rationality  in  game  theory,  American  Economic  Review  84:2,  1994,  pp.  391-395): 
Suppose  that  two  travelers  have  identical  luggage,  for  which  they  both  paid  the  same  price. 
Their  luggage  is  damaged  (in  an  identical  way)  by  an  airline.  The  airline  offers  to  recompense 
them  for  their  luggage.  They  may  ask  for  any  dollar  amount  between  $2  and  $100.  There  is 
only  one  catch.  If  they  ask  for  the  same  amount,  then  that  is  what  they  will  both  receive. 
However,  if  they  ask  for  different  amounts — say  one  asks  for  $m  and  the  other  for  $ m',  with 
m  <  m! — then  whoever  asks  for  $m  (the  lower  amount)  will  get  $(m  +  2),  while  the  other 
traveler  will  get  $(m  —  2). 

It  seems  at  first  blush  that  both  travelers  should  ask  for  $100,  the  maximum  amount,  for 
then  they  will  both  get  that.  However,  one  of  them  might  then  realize  that  he  is  actually  better 
off  asking  for  $99  if  the  other  traveler  asks  for  $100,  since  he  then  gets  $101.  In  fact,  $99  weakly 
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dominates  $100,  in  that  no  matter  what  Traveler  1  asks  for,  Traveler  2  is  always  at  least  as 
well  off  asking  for  $99  than  $100,  and  in  one  case  (if  Traveler  2  asks  for  $100)  Traveler  1  is 
strictly  better  off  asking  for  $99.  Thus,  it  seems  we  can  eliminate  100  as  an  amount  to  ask 
for.  However,  if  we  eliminate  100,  a  similar  argument  shows  that  98  weakly  dominates  99.  And 
once  we  eliminate  99,  then  97  weakly  dominates  98.  Continuing  this  argument,  both  travelers 
end  up  asking  for  $2!  In  fact,  it  is  easy  to  see  that  (2,2)  is  the  only  Nash  equilibrium.  With 
any  other  pair  of  requests,  at  least  one  of  the  travelers  would  want  to  change  his  request  if  he 
knew  what  the  other  traveler  was  asking.  Nevertheless,  in  experiments,  it  has  been  shown  that 
people  typically  ask  for  amounts  in  the  high  90’s;  very  few  people  ask  for  2.  | 

Pass  and  I  [28]  introduce  a  new  solution  concept,  iterated  regret  minimization ,  which  has  the 
same  qualitative  behavior  as  that  observed  in  the  experiments,  not  just  in  Traveler’s  Dilemma, 
but  in  many  other  games  that  have  proved  problematic  for  Nash  equilibrium,  including  the 
Centipede  Game,  Nash  bargaining,  and  Bertrand  competition.  As  the  name  suggests,  iterated 
regret  minimization  involves  the  iterated  deletion  of  strategies  that  do  not  minimize  regret.  It  is 
a  solution  concept  that  leads  to  quite  different  predictions  than  Nash  equilibrium,  while  being 
grounded  on  regret,  a  decision  rule  that  has  a  long  history  in  decision  theory,  and  has  a  great 
deal  of  support  from  the  psychology  literature. 

Roughly  speaking,  the  idea  of  regret  in  decision  theory  is  that  an  agent  chooses  an  action 
a  that  minimizes  regret  across  states,  where  the  regret  of  action  a  in  a  state  s  is  the  difference 
between  the  agent’s  utility  when  he  performs  a  in  a  state  s  and  when  he  performs  the  act  that 
gives  the  highest  utility  in  state  s.  To  apply  regret  in  a  strategic-form  game,  we  take  the  states 
to  be  the  other  players’  strategy  choices.  Iterated  regret  minimization  takes  this  idea  one  step 
further:  we  see  what  inferences  we  can  draw  starting  with  minimal  beliefs  about  the  other 
players’  strategies,  using  only  the  fact  that  the  other  players  are  regret  minimizer. 
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